Modeling and Diagnosing Domain Knowledge Using Latent Semantic Indexing

A Latent Semantic Index (LSI) was constructed from arguments made by Navy officers concerning events in an Anti-Air Warfare scenario. A model based on LSI factor values predicted level of domain expertise with 89% accuracy. The LSI factor space was reduced using MDS to five dimensions: aircraft route, aircraft response, kinematics, localization, and an unclassifiable element. Arguments in the localization category were reliably more common among officers with the greatest expertise. Automated classification of arguments into these elements achieved 84% accuracy. LSI may be a useful tool for automating aspects of modeling expertise and diagnosing knowledge deficiencies.